Many complex systems rely on highly specialised components in order to achieve their functionality or perform a particular role, and information tends to be handled using a combinations of physical hardware and graphics represented on a two-dimensional screen. Generally, the user interacts with such information provided on-screen using a mouse or touchscreen input method, whereas specific/specialised tasks tend to be effected through the use of bespoke switch panels, dials, buttons, slide controls, and the like. The result is a complex physical control environment, which places a relatively high workload and training burden on individuals, and increases the potential for human error during interaction.
Technological advances have addressed this issue to a limited extent, by, for example, the integration of LCDs into a single control element, thereby allowing the control element to perform multiple functions. Advances in touch screen and alternative technologies, such as voice commands and gaze recognition, have also attempted to alleviate some of the problems identified above. However, such technologies have severe limitations, particularly in relation to the types of control elements they can be used to replace or enhance, and the user is still, in any event, provided with a system with which they are required to interact in a ‘conventional’ manner, i.e. as if the physical controls were still present and in the same relative locations as in a system comprised entirely of physical controls. Thus, the user is still constrained, to a large extent, by conventional methods of interacting with the system, with the result that the above-mentioned disadvantages are still present.
It would, therefore, be desirable to provide a method and apparatus for controlling the functions and/or operations of a system which at least addresses some of the problems outlined above.
Virtual reality systems are known, comprising a headset which, when placed over a user's eyes, creates and displays a three-dimensional virtual environment in which a user feels immersed and with which a user can interact in a manner dependent on the application. For example, in some prior art systems, the virtual environment created may comprise a game zone, within which a user can play a game. However, in an environment where the user needs to be able to see where they are going in order to function appropriately within their physical environment, such systems are unsuitable.
More recently, augmented and mixed reality systems have been developed, wherein an image of a real world object can be captured, rendered and placed within a 3D virtual reality environment, such that it can be viewed and manipulated within that environment in the same way as virtual objects therein. Other so-called augmented reality systems exist, comprising a headset having a transparent or translucent visor which, when placed over a user's eyes, creates a three-dimensional environment with which the user can interact, whilst still being able to view their real environment through the visor.
However, in an augmented reality environment, whereby the user can “see” all aspects of their real world environment through the visor as well as the multiple sources of data in the virtual environment, the resultant 3D environment becomes excessively cluttered and it becomes difficult for a user to focus on the important elements thereof. Furthermore, this does not address the problem outlined above, whereby in complex systems, multiple physical controls are provided in fixed locations within the real world environment, which do not enable the user to interact with them conveniently or intuitively.
It is therefore an object of aspects of the present invention to address at least some of these issues.